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(54) Confocal spectroscopy system and method 

(57) A confocal spectral imaging system comprises 
a light source (110), a light modulator (120) forming an 
illumination aperture and directing an illumination pat- 
tern to conjugate object locations, and analyzing means 
(150) with a detection aperture, dispersive elements 
and a detector (156), wherein the illumination and 
detection apertures are in conjugate optical planes, and 
the light modulator consists of an array of light modula- 
tor elements, a group of which being arranged accord- 
ing to the illumination pattern and forming the 
illumination aperture, and are controlled such that the 
illumination pattern is directed to time-dependent 
changing conjugate locations of the object. A program- 
mable light source comprises a white light source, dis- 
persion means and a spatial light modulator with an 
array of individually time-dependent controllable modu- 
lator elements being illuminated with the dispersed light 
and providing a position selective transmittivity or reflec- 
tivity, so that a light with a predetermined wavelength 
distribution passes the light modulator. 
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Description 

[0001] The invention generally relates to a confocal 
spectroscopic imaging system (CSIS) employing a con- 
focal spectroscopic system (CSS) and/or a wavelength 
progammable light source (PLS). The components CSS 
and PLS can be used separately or together to create 
the CSIS. The invention particularly relates to a pre- 
ferred implementation of the CSIS with a programmable 
array microscope (PAM) in which the patterns of illumi- 
nation and detection are freely programmable. 
[0002] Confocal microscopy based on point scanning 
systems with conjugate pairs of illumination and detec- 
tion apertures is an effective tool for imaging a micro- 
scopic object to be investigated with direct optical 
sectioning. The discrete aperture spots are illuminated 
in the object plane of the microscope from which 
reflected or fluorescent light is observed through the 
conjugate detection apertures in an image plane. Com- 
monly used confocal microscopes based on scanning 
systems with mechanically translated aperture disks 
(so-called Nipkow disks with a plurality of apertures) or 
with rotating mirrors being adapted to scan an object 
with a laser beam (Confocal Laser Scanning Micros- 
copy, CSLM). Disk-based scanning systems have limi- 
tations as to a restriction of the illumination field, a 
degraded contrast and high intensity losses. Typically 
less than 3% of the aperture disk is transmissive since 
the spacing between the pinholes must be large to 
maintain the confocal effect. On the other hand, the 
CSLM suffers from a low duty cycle imposed by the 
sequential single point mode of data acquisition. 
[0003] A modified spatially light modulated micro- 
scope has been described by M. Liang et al. in "Optics 
Letters" (vol. 22, 1997, p. 751-753) and in the corre- 
sponding US patent No. 5 587 832. A two-dimensional 
spatial light modulator is formed by a digital micromirror 
device (in the following: "DMD") which reflects illumina- 
tion light from a source (laser or white light source) to a 
probe and detection light from the probe to a two-dimen- 
sional detector. Each micromirror of the DMD is individ- 
ually controlled so as to form an illumination and 
detection spot or not. 

[0004] A general disadvantage of the prior art confocal 
microscopes concerns the detection of spectrally 
resolved images. The duty cycles and the illumination 
and detection intensities are limited so that a spectral 
imaging is not possible within practically interesting 
measuring times. Furthermore, excitation sources with 
rapid selection of large numbers (>50) of multiplexed 
spectral elements at kHz-frequencies are not available. 
[0005] Hadamard transform spectrometers (in the fol- 
lowing: "HTS") are known which provide a positional 
and spectral selectivity on the basis of a detection 
through a so-called Hadamard encoding mask. Such 
spectrometers are described e. g. by R. M. Hammaker 
et al. in "Journal of Molecular Structure" (vol. 348, 1 995, 
p. 135-138) or by P. J. Treado et al. in "Applied Spectros- 



copy" (vol. 44, 1990, p. 1-5, vol. 44, 1990. p. 1270- 
1275). 

[0006] A Hadamard transform Raman microscope 
according to P. J. Treado et al. is shown in Figures 7A,B 

5 as HTS 70. Figure 7A shows the arrangement of a Had- 
amard mask 72 in the detection path of a microscope 
71. The Hadamard mask 72 is a linear encoding mask, 
and a spatial multiplexing of spectral measurments is 
obtained by driving the mask with a translation stage 73. 

10 The image obtained with the spectrograph 74 is 
decoded with a control computer 75. Details of the con- 
ventional microscope 71 used in the HTS 70 are shown 
in Figure 7B. 

[0007] The HTS according to P. J. Treado et al. has the 
is following disadvantages. The illumination is restricted to 
a certain excitation wavelength without any controllabil- 
ity. The arrangement allows only Raman measure- 
ments. Because of the use of a conventional 
microscope, the generation of optical sections of an 
20 object under investigation without image processing is 
impossible. The mask is a non-programmable light 
modulator only. 

[0008] Accordingly, the application of known HTS is 
generally restricted to optical detection paths or to acou- 
25 sto-optical measurements, Raman microscopy and con- 
ventional microscopy. 

[0009] A confocal Raman microspectroscopy system 
is described by G. J. Puppels et al. in "Nature" (vol. 347, 
1990, p. 301-303). Light scattered by an object under 

30 investigation is collected by an objective and coupled 
through a pinhole for confocal detection into a spec- 
trometer. The detection through a pinhole represents an 
essential limitation of the measured light intensity so 
that the applicability of the known microspectroscopy 

35 system is restricted. 

[001 0] Confocal line scanning systems are commonly 
known. The work of P. A. Benedetti et al. in "Journal of 
Microscopy" (vol. 165, 1992, p. 119-129) discloses the 
use of a spectroscopic system with the confocal-line 

40 (CL) method. The CL technique has the following disad- 
vantages. First, the formation of the line-shaped illumi- 
nation needs extended optical components (slits, 
lenses, mirrors) with strict adjustment requirements. 
Second, the optical components are fixedly positioned, 

45 so that the scanning perpendicular to the line direction 
has to be performed by a mechanical object scanning 
table. Accordingly, the scanning speed is strongly 
restricted and particularly the spectral imaging is time 
consuming. Third, the illumination in this CL system is 

so restricted to the use of one slit only. An operation with a 
multiplexing in a spatial domain is impossible. 
[0011] Real-time confocal microscopy or imaging, in 
particular in the field of imaging biological objects like 
cells or parts thereof, calls for further improvements with 

55 regard to high contrast, sensitivity, detection speed and 
for an extended applicability of spectral imaging. 
[0012] It is the object of the present invention to pro- 
vide an improved device and method for confocal spec- 



EP 0 916 981 A1 



4 



tral imaging allowing rapid data acquisition, in particular 
with effective optical sectioning, high spatial resolution, 
high optical efficiency and spectral resolution. It is a par- 
ticular object of the invention to provide rapid imaging of 
biological or chemical materials (e. g. living cells, reac- 5 
tion composites etc.) while exploiting the ability of spec- 
troscopic measurements and thus information about 
molecular structure and function. Due to its inherent 
sensitivity and selectivity, molecular fluorescence is a 
preferred spectroscopic phenomenon to be imple- to 
mented with the new imaging device and method. It is a 
further object of the invention to provide a programma- 
ble light source adapted to generate a light output with a 
wavelength or wavelengths rapidly selectable according 
to a predetermined sequence. 75 
[001 3] The above object is solved by a confocal imag- 
ing device or method or by a light source comprising the 
features of claim 1, 11 or 18, respectively. Advanta- 
geous embodiments of the invention are defined in the 
dependent claims. 20 
[0014] The basic idea of the inventors is the operation 
of a confocal optical imaging system having spectro- 
scopic analyzing means with spatial light modulator 
means (in the following: SLM). The SLM means have an 
array of light modulator elements. One array can be a 25 
mask consisting of modulator elements each of which is 
individually controllable. The controllability relates to 
optical features of the elements, as transmission, reflec- 
tion or diffraction. A changing group or all of the modu- 
lator elements form an coding illumination pattern as an 30 
illumination and/or detection aperture. The patterns for 
illumination and detection matched to each other are 
generated separately or by a common modulator. The 
modulator elements representing the actual aperture 
are selectable according to a predetermined pattern 35 
sequence, so that an illumination pattern sequence is 
formed in the conjugate object plane and changing loca- 
tions are illuminated. The illumination pattern sequence 
allows a time encoded illumination. Accordingly, the 
SLM means form an illumination and/or detection cod- 40 
ing means. With a modified version, the array can be a 
movable mask and the sequence is obtained from a 
mask movement. In this case, an imaging spectrograph 
is placed in the conjugate image plane of the micro- 
scope and multiple apertures of the mask are used for 45 
collecting the light. The system can contain a further 
array of light modulator elements being the mask of a 
spectrally programmable illumination source described 
below. 

[001 5] The illumination pattern sequence comprises a so 
systematically shifting line or lines either spaced regu- 
larly, according to a pseudo-random sequence of finite 
length, according to a S-matrix type Hadamard 
sequence or randomly; regular dot lattices as e. g. trian- 
gles, squares, rectangles, hexagons; random patterns 55 
or pseudo-random patterns of finite length based on so- 
called Walsch, Sylvester, Hadamard or Golay 
sequences; square or rectangular grids formed from 



intersecting line patterns; plane filling patterns being 
adapted to turn on the SLM elements an integral 
number of times when sequentially produced on the 
SLM; or a combination of the above pattern sequences 
on the basis of logic Boolean AND, OR, NAND, NOR or 
XOR operations. 

[0016] According to preferred embodiments, the 
detection aperture coincides with the illumination aper- 
ture and/or forms the entrance of spectrally dispersive 
elements of the analyzing means. When the entrance of 
the spectrally dispersive elements and a two-dimen- 
sional detector camera are arranged in conjugate opti- 
cal planes, the provision of an entrance slit can be 
omitted. 

[0017] Preferably, the confocal spectroscopy system 
of the invention is implemented with a PAM as 
described in the European patent application No. 
97118354.6 filed on October 22, 1997, the complete 
contents of which is included to the present specifica- 
tion by reference. 

[0018] Light source means of the system contain a 
white light lamp with a fixed or tunable wavelength filter 
or a laser. A white light lamp with a tunable wavelength 
filter is obtainable in particular from a spectrally pro- 
grammable illumination source being an independent 
subject of the invention. In this case, the wavelength fil- 
ter comprises a spectrally dispersive element and a 
two-dimensional array of light modulator elements being 
adapted to select at least one of the predetermined 
wavelengths passed through the dispersive element. 
[0019] According to preferred applications of the 
invention, the detection light is fluorescence, phospho- 
rescence, reflected or Raman scattered light emitted 
from the object. 

[0020] The confocal spectral imaging method of the 
invention comprises the steps of focusing illumination 
light from light- source means via a light modulator 
means with an array of light modulator elements forming 
an illumination pattern in conjugate locations to an 
object under investigation and collecting light emitted 
from the object with analyzing means with a detection 
aperture, spectrally dispersive elements and a two- 
dimensional detector camera, wherein the illumination 
aperture and the detection aperture are placed in conju- 
gate optical planes and wherein the light modulator 
means are controlled in a time-dependent fashion such 
that a group of modulator elements shaped according to 
a predetermined aperture or pattern forms an illumina- 
tion pattern sequence. For the collection of a two- 
dimensional optically sectioning image, the pattern 
sequence is used as featured above. The time-depend- 
ent control is adapted to provide a scanning of the 
object plane for collecting the two-dimensional image as 
an optical section. 

[0021 ] Preferably, the spectral imaging is implemented 
in using a programmable array microscope as 
described in the European patent application No. 
97118354.6 filed October 22, 1997 to create confocal 
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arrangement of illumination and detection apertures. In 
this preferred arrangement, two conplete spectral 
images are recorded corresponding to the conjugate l c 
and non-conjugate Inc images outlined in the above 
mentioned application. A deconvolution step can then 5 
be applied to the acquired images to reduce blurring in 
one or more optical sections. The implementation of the 
blurring matrix is given with a further spectral or time 
dimension. 

[0022] Advantageous applications of the invention 10 
comprise methods of simultaneous position selective 
initiating and monitoring photochemical reactions on a 
substrate, wherein the initiating includes the irradiation 
of predetermined substances with a suitable wave- 
length (e. g. with a programmable light source as is 
described below) and the monitoring includes fluores- 
cence measurements with the substances or the reac- 
tion products; investigations in the cell and tissue 
biology; analytical/biotechnological procedures, in-situ 
hybridization in genetic analysis; the formation of an 20 
optical mask for position selective photochemical reac- 
tions; position selective fluorescence measurements 
including excitation and emission spectra of one or 
more intrinsic or external fluorophores, lifetime meas- 
urements and/or polarization measurements; readout of 25 
biochemical arrays on chips; large scale surface inspec- 
tions in the semiconductor industry, and/or optical 
recording and read-out. 

[0023] The invention has the following advantages. 
First, the sectioning of samples combined with spec- 30 
troscopy gives a better indication of the spectra present 
at a particular position in a sample, in biology this will 
give better discrimination of co-localized structures. 
Second, the invention allows for selective photochemis- 
try with detection of the products in a confocal mode. 35 
Third, the invention allows the ability to do spectrally 
resolved lifetime measurements if combined with a 
phase modulation or time correlation techniques. The 
imaging allows a spectral discrimination against auto- 
fiuorescence and other interfering signal sources. The to 
invention allows a combination with 2- or multi-photon 
excitation in fluorescence measurements and with ster- 
eoscopic imaging. 

[0024] Further details and advantages of the invention 
are described in the following with reference to the 45 
attached drawings which show in: 

Fig. 1 : a schematic view of a first embodiment of a 
spectral imaging CSIS according to the invention 
consisting of a CSS only; so 

Figs. 2a to 2f: illustrations of spectrally resolved 
measurements obtained with the arrangement of 
Figure 1 ; 

55 

Fig. 3: a schematic view of second embodiment of 
a spectral imaging CSIS according to the invention; 



Fig. 4: a schematic view of third embodiment of a 
spectral imaging CSIS according to the invention; 

Fig. 5: a schematic view of a programmable light 
source according to the invention; 

Rg. 6: a schematic view of a CSIS according to the 
invention incorporating both a confocal spectro- 
scopic system (CSS) and a programmable light 
source (PLS); and 

Rg. 7: a schematic view of a Hadamard transform 
Raman microscope using a non-programmable 
light modulator mask in the detection path only 
{prior art). 

[0025] In the following, two separate aspects of the 
invention concerning the spectral imaging and the pro- 
grammable light source are described. With regard to 
the application of a system according to the invention, 
the steps of object handling, image processing and dis- 
playing are not described in detail as far as they are 
known from common confocal scanning devices. 

Spectral imaging 

[0026] The confocal spectroscopic system (CSS) with 
spectral imaging according to the invention can be 
implemented on the basis of SLMs being operated in 
transmission (e. g. programmable aperture masks on 
the basis of liquid crystals or micromechanical 
switches), in reflection (e. g. DMD), with diffraction, or 
with lithographically etched modulators. The CSS con- 
tains a spatial light modulator which is placed in the 
image plane of the microscope. A group or all of the ele- 
ments of the spatial light modulator each of which are 
conjugate to a distinct point in the focal plane of the 
microscope objective, represent an illumination aper- 
ture and define a programmable array which may be 
used for illumination and/or detection. The position of 
the programmable array with regard to the optical axis 
of the illumination path is changed according to an indi- 
vidual control of each of the modulator elements or 
according to a physical movement of a modulator with a 
fixed pattern. According to a particular embodiment of 
the invention, the CSS contains another spatial light 
modulator being a wavelength selective device adapted 
to illuminate the above first light modulator as a part of 
a light source as described below. 
[0027] The CSIS 100 of Rgure 1 representing an 
implementation of the invention in a microscope using a 
spatial light modulator 120. It contains an optional two 
dimensional camera system 160 for integrating it into a 
PAM. This CSIS is a dual path single reflection design. 
The "dual path" refers to the fact that illumination and 
detection follow two separate optical paths. Further, 
only a single reflection direction of the reflective SLM is 
used. An alternative arrangement is a common path 
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double reflection design shown in Figure 4. The CSIS 
100 basically includes light source means 1 10 which is 
a white lamp as shown (with an excitation wavelength 
selection means) or a source with fixed or tunable wave- 
lengths like a laser source or a programmable illumina- 5 
tion source described below, a DMD 120 operating as a 
SLM being arranged in an image plane of the micro- 
scope, imaging optics 130, a probe portion 140, analyz- 
ing means 150 and a full field detection means 160. 
[0028] The DMD 120 consists of an array of square 10 
mirrors each of which can be separately tilted into two 
stable resting states over a predetermined tilt angle ± a 
to the normal. The value of the tilt angle a depends on 
the DMD practically used. Each mirror element has a 
predetermined area which again depends on the DMD. 
Preferred parameters are e. g. a = ± 10° with < 20 urn 
square mirrors. Each mirror element can be switched 
between the resting states with characteristic frequen- 
cies in the kHz-range according to a predetermined 
modulation pattern. The resting states are called "on rt - 
and "off-positions according to the respective pattern 
(e.g. line) shape contributions ("on' 1 ) focused to conju- 
gate positions of the object and the remaining illumina- 
tion light CofT). 

[0029] The DMD 120 is illuminated with the light 
source means 110 via a lens 11 1. The excitation filter 
131 can be a bandpass for a predetermined wavelength 
range of interest. For the detection of excitation spectra, 
the bandpass can be implemented by a variable disper- 
sive element. The reflections from the n on n -elements 
are focused via the first beam splitter 132 (semitrans- 
missive mirror) and the objective 141 into the object 
plane of the object 142. The beam splitter 132 can 
include a dichroic mirror. Fluorescence or scattered light 
stimulated in the object plane returns through the objec- 
tive 141 to the first beam splitter 132 where it passes 
along the optical axis via a filter 153, a tube lens 152 
and a second beam splitter 151 to the analyzing means 
150. The confocal arrangement allows to reject light 
from the non-conjugate positions of the object. The filter 
153 is preferably a long pass filter being adapted for the 
fluorescence, phosphorescence or Raman scattering 
measurement (A^ > Xq). In the analyzing means 150, 
the detection light is collected and the obtained image 
signals are processed for image or optical section 
reconstruction. 

[0030] CSIS 100 can be implemented with a Digital 
Light Processing kit (e. g. of Texas Instruments, Dallas, 
USA) as a light source in combination with a micro- 
scope with side illumination ports (e. g. Zeiss Axioplan) 
and a camera module with a CCD camera (e. g. CH220 
camera of Photometries, Tucson, USA, with Kodak 
KAF1400 CCD sensors). The spectrograph 154 of the 
analyzing means 150 is any conventional spectrograph 
as e. g. a SpectraPro-150 imaging spectrograph with a 
300 lines/mm grating, Acton Research Corp., Acton, 
USA, and a SBIQ ST-6 camera, Santa Barbara Instru- 
ments Group, Santa Barbara, USA. The entrance slit is 



adapted to the line width generated with the DMD 120, 
e. g. according to the above example the slit is set to 25 
jim. The probe portion 140 contains a driving device 
143 for z-displacements (perpendicular to the focus 
plane) like a computerized focus control e. g. of Ludl 
Electronics Products, Hawthorne, USA. The objective is 
e. g. a 40 x 1 .3 NA oil immersion objective. Each system 
of the invention (e. g. the CSIS 100) additionally con- 
tains a driving unit for controlling the modulator ele- 
ments or a movable mask, a control unit, calculation, 
decoding and/or deconvolution circuits and display units 
(not shown). The illumination path is shown with a 
hatched area. The control unit is particularly adapted to 
collect camera signals in synchronization with the con- 
trol of the modulator elements. 
[0031] For confocal spectral imaging, a moving pat- 
tern sequence is written to the DMD 120 causing light 
from the corresponding mirrors to be collected with the 
illumination tube lens 133 and directed through the filter 
131. For obtaining a complete two-dimensional image 
of the object, the pattern is to be scanned, e. g. each of 
the modulator elements is to be controlled separately 
such that the position of the pattern changes according 
to a predetermined pattern sequence. The DMD 120 is 
controlled as described above e. g. such that the object 
to be investigated is illuminated e. g. with a line pattern 
of SLM elements. This situation is called line confocal 
operation in which the line of pixels in the SLM conju- 
gate to the entrance slit of the spectrograph are turned 
"on". With a DMD mirror element width of about 16 fim 
and a 40x objective, the illumination line imaged to the 
object has a width of about 400 nm. This line is placed 
conjugate to the entrance slit of the analyzing means. 
[0032] The imaging pattern sequence generally may 
comprise a fixed or systematically shifting line or lines 
either spaced regularly, according to a pseudo-random 
sequence of finite length, according to a S-matrix type 
Hadamard sequence or randomly, regular dot lattices as 
e. g. triangles, squares, rectangles, hexagons, random 
patterns or pseudo-random patterns of finite length 
based on so-called Walsch, Sylvester, Hadamard or 
Golay sequences. Sylvester matrices are preferred 
encoding sequences due to the ability to multiplex one 
spatial domain allowing a two dimensional detector to 
record three dimensions of data. This improves the 
SNR of the measurements in many cases. The line 
imaging (confocal slit illumination) is preferred due to 
the adaptation to the shape of the entrance aperture of 
commonly used spectrographs. 
[0033] In the example, a demagnif ied image of the line 
is formed by the objective 1 41 in the object plane where 
the fluorescence is stimulated or the light is scattered. 
The emitted light forms a line image at the entrance slit 
155 of the imaging spectrograph 154. Confocal (in- 
fbcus) light passes through the slit 155 which is 
recorded with the one- or two-dimensional camera 156. 
The further processing of the collected image corre- 
sponds to known decoding and reconstruction proce- 



20 



25 



30 



35 



40 



45 



50 



5 



9 



EP 0 916 981 A1 



10 



dures. 

[0034] For full field imaging, the entire DMD 120 is 
turned to the "on"-position. For spectral imaging, the 
optical path is similar except that the light passing 
through the entrance slit 155 contains a greater amount 
of out-of-focus light from structures above and below 
the object plane. For non-spectral imaging, the full field 
additionally is recorded with the detecting means 160 
containing a two-dimensional camera 161. The second 
camera 161 collects the full field image via the second 
beam splitter 151. This allows the direct comparison 
between confbcal and conventional operation modes at 
identical probe positions. The camera 161 can particu- 
larly be used in a so-called single camera mode as 
described in the copending European patent application 
No. 971 1 8354.6 filed on October 22, 1 997 so that a con- 
ventional and a confocal image are obtained alternately. 
[0035] Furthermore, excitation spectra can be 
obtained by arranging a wavelength selector in the opti- 
cal path between the light source and the SLM. Accord- 
ing to an alternative application, absorption spectra in a 
reflection mode are collected. The SLM is preferably 
controlled with a coding sequence according to the 
Hadamard sequences as cited above. Accordingly, the 
Hadamard sequences are used for the first time for both 
the illumination and the detection simultaneously. 
[0036] Figures 2a-f show examples of spectral images 
obtained with an arrangement according to Figure 1. 
For print technical reasons, colored photographs are 
shown only schematically. 

[0037] Figure 2a shows an example of a f luorescently 
labelled biological specimen. A 15 jmm section of Con- 
vallaria rhizome having a high spatial frequency struc- 
ture was stained with acridine orange. Upon excitation 
at 450 - 490 nm, the fluorescence is emitted in two 
spectral regions yellow and red which have different 
gray values in the figure. The arrow of Figure 2a indi- 
cates the position of the entrance slit. The horizontal 
lines refer to the corresponding spectra in Figures 2b, 
2c. Figure 2b shows a full field (conventional) spectral 
line image from the slit position in Figure 2a. The sche- 
matic^ drawing shows the fluorescence maximum in 
the red range which is dominant in the lower part of the 
figure according to the corresponding structure in the 
probe. In the middle and upper parts of the figure, the 
red fluorescence overlaps to geometrical ranges where 
changed spectral properties are expected on the basis 
of Figure 2a. This overlapping represents a reduced 
resolution of the conventional imaging. 
[0038] Figure 2c shows a confocal spectral image of 
the same slit position as in Figure 2b. In the upper part 
of the figure, the regions wfth red fluorescence accord- 
ing to red emitting structures crossed by the horizontal 
lines in Figure 2A are clearly separated from the low 
emitting "shadows" in Figure 2a. The overlapping as in 
Figure 2b is completely avoided. This represents an 
essential advantage of the invention as to the improved 
contrast and spectral purity of the images. 



[0039] Figure 2d shows the comparison of the inten- 
sity distribution along the slit axis at 540 nm (normalized 
to the total light collected) for confbcal (above) and full 
field imaging (below). According to the structure in Fig- 
5 ure 2a, the stronger fluorescing elements yield maxima 
in the intensity curves. The confocal image in Figure 2d 
shows an improved background rejection compared 
with the full field image. 

[0040] The fluorescence spectra of Figures 2e and 2f 

10 are taken at the respective positions e, f indicated in Fig- 
ure 2a. The spectra are uncorrected with regard to the 
quantum efficiency of the detector. Figure 2e (normal- 
ized as in Figure 2d) corresponds to the "dark" structure 
e in Figure 2a. The conventional spectrum shows a rel- 

75 ative strong fluorescence caused by emissions around 
the structure e. On the other hand, the confocal spec- 
trum is suppressed as the interfering emissions are 
avoided. This result yields a background suppression by 
a factor of about 3. Figure 2f (normalized to the intensity 

20 of 546 nm) shows a spectrum corresponding to the 
structure f in Figure 2a. The spectrum is shifted to the 
red in the confocal case due to the less interference 
from the adjacent yellow structure. 
[0041] Figure 3 shows another embodiment of a 

25 microscope having an integrated confocal illuminator 
and spectrograph. The CSIS 300 represents an inte- 
grated confocal illuminator and spectrograph. 
[0042] On the optical axis of the CSIS 300 a probe 
portion 340, imaging optics 330a and 330b and analyz- 

30 ing means 350 with an imaging spectrograph 354 are 
arranged. The optics 330b is an anamorphic system for 
coupling the light into the entrance of the spectrograph. 
The anamorphic system contains a component, e. g. a 
cylindrical lens, to selectively demagnify (compress) a 

35 single optical axis. Contrary to the embodiment of Fig- 
ure 1, also the SLM 320 is arranged axially defining 
simultaneously the illumination pattern (illumination 
aperture) and the entrance slit (detection aperture) of 
the analyzing means 350. The traditional slits of the 

40 imaging spectrograph 354 are removed or, alternatively, 
opened sufficiently to allow a virtual image of the SLM 
320 to be reproduced at the entrance plane 355 indi- 
cated in the figure with VSLM ("virtual image of SLM"). 
[0043] The illumination light from the light source 

45 means 3 1 0 is directed via an excitation filter 3 1 1 and an 
illumination lens 331 through a side port of the micro- 
scope and a beam splitter 332 (e. g. half-silvered mirror) 
to the SLM 320. The SLM 320 is a transmissive device 
wherein each element is controllable to change the 

so transmittivity for passing the illumination light and, after 
excitation of the probe, the detection light which passes 
through the imaging optics 330 (including a tube lens 
and an emission filter) to the entrance plane 355 of the 
spectrograph 354 with the detector camera 356. The 

55 spectrograph 354 is illustrated with the Fastie-Ebert 
design having focusing mirrors F1 , F2 and a grating G1 , 
however a wide range of other dispersive arrangements 
including prisms, holographic gratings or acousto-opti- 
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cal tunable filters (AOTF's). The dashed line defines the 
boundaries of the spectrograph. 
[0044] The imaging procedure is conducted as 
described above for the first embodiment. In particular, 
the SLM 320 is controlled e. g. such that the object to be s 
investigated is illuminated with a line pattern of SLM ele- 
ments or any other appropriate illumination pattern. 
[0045] The arrangement of Figure 3 can be modified 
by providing a second camera for forming a full field 
imaging leg as in Figure 1 . Furthermore, the filters can to 
be replaced by AOTF's. The SLM pattern can be 
selected in any form as described above. The beam 
splitter 332 can include a dichroic mirror. 
[0046] The SLM 320 can be a mask with a fixed pat- 
tern produced with lithographically etching or with a 15 
photographic film. The encoding mask used for both illu- 
mination and detection is combined with a support 
adapted to physically move or translate the mask to 
generate a series of patterns needed for spectra! imag- 
ing. In the simplest case the pattern could be a line pat- 20 
tern as shown in the upper inset "field V that is shifted 
sequentially. Alternatively, an encoding set of patterns 
(e. g. lines or apertures or more complicated patterns) 
are formed on the mask as shown in the lower inset 
"field 2". "Field 2" shows a "stylized" pattern approximat- 25 
ing a Hadamard pattern. In this case, after the data are 
collected, a Hadamard transformation can be applied to 
the data and the image recovered. 
[0047] A microscope 400 comprising similar compo- 
nents as the above embodiments but having a common 30 
path double reflection design is shown in Figure 4. Illu- 
mination light from the light source means 410 is 
directed via the beam splitter 432 and a mirror 421 to 
the DMD 420 being controlled as described above. The 
DMD 420 illuminates the object and reflects emitted 35 
detection light back to the analyzing means 450. Addi- 
tionally to the spectroscopy leg of the CSIS 400, a so- 
called imaging leg 460 can be arranged which com- 
prises components for conventional and/or confocal 
imaging of the object under investigation. The imaging 40 
can be performed as described in the copending Euro- 
pean patent application No. 97118354.6 filed on Octo- 
ber 22, 1997 the contents of which is included to the 
present specification by reference. 

45 

Spectrally programmable light source (PLS) 

[0048] Another subject of the invention is the provision 
of a spectrally programmable light source (PLS) being 
adapted for rapid spectral shaping of an incident light, so 
This PLS allows an advantageous combination with the 
above confocal spectral imaging technique as well as 
general applications in ail fields of illumination for meas- 
urement or display purposes. A preferred application is 
the measurement of high throughput excitation spectra 55 
by modulating the spectrum leaving the PLS according 
to a Hadamard sequence. Another preferred application 
is the use of a PLS as a component (light source 



means) in a confocal spectroscopy system described 
above (see Figure 6). 

[0049] The PLS comprises a white or broad band light 
source, focusing optics, a dispersion element, a spatial 
light modulator and a device for recombining the dis- 
persed wavelengths. A schematic illustration of such a 
system is shown in Figure 5. The light source 510 is dis- 
persed with the dispersion element 515 (prism or grat- 
ing arrangement or similar spectrograph arrangements) 
over the surface of a spatial light modulator as 
described above. Prism optics have the advantage of an 
increased optical efficiency. 

[0050] The spatial light modulator (structured as 
described above) is a transmission modulator 520 with 
individually controllable modulator elements. However a 
corresponding reflection or diffraction arrangement is 
also possible. According to the position selective trans- 
missivity of the transmission modulator 520 (formation 
of "windows" with a certain outer shape), particular 
wavelengths pass through the modulator e. g. to a 
device for recombining the transmitted rays or directly to 
a probe. The outer shape of the selective transmiss'rvity 
of the modulator (e. g. a slit shape) can be adapted to 
the shape of the light output from the dispersion ele- 
ment (e. g. a slit). The modulator elements are control- 
led such that groups of modulator elements are 
transmissive or reflective and/or absorbing simultane- 
ously which form an outer shape corresponding to the 
shape of the light output. If this shape is e. g. a slit or line 
shape, the modulator elements are patterned accord- 
ingly as a line. The modulator can be controlled in a 
dynamic manner, so that the wavelength specific modu- 
lator output changes its wavelength characteristic 
according to a predetermined time pattern with high 
rates. 

[0051] In the illustrated case, the device 530 for 
recombining is an integrating sphere. Other alternatives 
are arrangements collecting the light with a lens system 
and using it for the illumination in parallel mode (Kohler 
type illumination), using a matching set of dispersion 
optics as used by Tillota et al. in "Applied spectroscopy", 
vol. 41 , 1987, p. 727-734, using fiber optics or liquid light 
conduits or using scattering surfaces (ground glass). 
[0052] The device for recombining can be omitted so 
that the light passing through the modulator illuminates 
a probe with a complex shifting pattern with different 
wavelengths. Preferably, the modulator elements are 
controlled such that the transmission or reflection pat- 
tern corresponds to rows or columns of an S-type Had- 
amard coding matrix. Alternatively, the modulator 
elements are controlled such that the output wave- 
lengths follow a predetermined time sweep, e. g. a time 
sweep according to a linear or other function. 
[0053] The lower part of Figure 5 shows the effect of 
the modulator. While the input spectrum (bottom left) 
shows an essentially "white" distribution, the output 
spectrum has a predetermined shape. The shape is lim- 
ited by the number of resolution elements in the disper- 
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sive element and the extent of the dispersion. 
[0054] Figure 6 shows a CSIS 600 combining both a 
confocal spectroscopic system (CSS) as described with 
reference to Figures 1 , 3 or 4 and a wavelength progam- 
mable light source (PLS) as described with reference to s 
Figure 5. The object is illuminated through a plurality of 
coding mask modulators 611, 620. The CSIS contains 
as a light source 610 the PLS with a first spatial light 
modulator 61 1 adapted to select a fixed excitation wave- 
length or to modulate the excitation wavelength. In the 10 
latter case, the excitation wavelength is modulated 
according to an appropriate coding sequence for obtain- 
ing position and spectrally resolved excitation spectra, 
e. g. according to a Hadamard sequence. The CSS con- 
tains a second spatial light modulator 620 forming the 15 
illumination pattern as described above. Further details 
and the function of the CSIS 600 correspond to the 
above embodiments. 

Claims 20 

1. Confocal spectral imaging system (100, 300, 400, 
600) comprising: 

- light source means (110, 310, 410, 610); 25 

- light modulator means forming an illumination 
aperture and being adapted to direct a prede- 
termined illumination pattern to conjugate loca- 
tions of an object under investigation; and 

- analyzing means (150, 350, 450, 650) with a 30 
detection aperture, spectrally dispersive ele- 
ments and a detector camera, 

wherein the illumination and the detection 
apertures are placed in conjugate optical 
planes, 35 
characterized in that 

the light modulator means (120, 320, 420. 620) 
consist of a two-dimensional array of light mod- 
ulator elements a group of which being 
arranged according to the illumination pattern 40 
and forming the illumination aperture, wherein 
the light modulator means are controllable 
such that the illumination pattern is directed to 
time-dependent changing conjugate locations 
of the object 45 

2. System according to claim 1 , wherein the light mod- 
ulator elements are individually controllable such 
that the reflective, transmissive or diffractive proper- 
ties thereof are changed. so 

3. System according to claim 1 or 2, wherein the light 
modulator means are adapted to scan at least a 
part of the object according to a predetermined illu- 
mination pattern sequence. 55 

4. System according to claim 3, wherein the pattern 
sequence is selected to be a systematically shifting 



line or lines either spaced regularly, according to a 
pseudo-random sequence of finite length, accord- 
ing to a S-matrix type Hadamard sequence or ran- 
domly; regular dot lattices as e. g. triangles, 
squares, rectangles, hexagons; random patterns or 
pseudo-random patterns of finite length based on 
so-called Walsch, Sylvester, Hadamard or Golay 
sequences; square or rectangular grids formed 
from intersecting line patterns; plane filling patterns 
being adapted to turn on the SLM elements an inte- 
gral number of times when sequentially produced 
on the SLM; or a combination of the above pattern 
sequences. 

5. System according to one of the foregoing claims, 
wherein the detection aperture is formed by the pat- 
tern shaped group of modulator elements so that 
the conjugate optical planes for illumination and 
detection coincide. 

6. System according to one of the foregoing claims, 
wherein the entrance of the spectrally dispersive 
elements and the two<limensional detector camera 
are arranged in conjugate optical planes. 

7. System according to one of the foregoing claims, 
wherein the entrance plane of the spectrally disper- 
sive elements is defined by the two-dimensional 
array of light modulator elements. 

8. System according to one of the foregoing claims, 
being a part of a confocal programmable array 
microscope and further comprising full field detec- 
tion means (160) with a two-dimensional camera for 
collecting a conjugate and/or a non-conjugate 
microscopic image of the object. 

9. System according to one of the foregoing claims, 
wherein the light source means contain a white tight 
lamp and a fixed or tunable wavelength filter which 
comprises a spectrally dispersive element and 
another two-dimensional array of individually time- 
dependent controllable light modulator elements 
being illuminated with dispersed light from the 
spectrally dispersive element and providing a posi- 
tion selective transmittivrty or reflectivity for produc- 
ing light with a predetermined wavelength 
distribution. 

10. System according to one of the foregoing claims, 
wherein the detection light is fluorescence, phos- 
phorescence, reflected light or Raman scattered 
light emitted from the object. 

11. Confocal spectral imaging method, comprising the 
steps of: 

- focusing illumination light from light source 
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means (110, 310, 410, 610) via light modulator 
means which form an illumination aperture and 
shape a predetermined illumination pattern, to 
conjugate locations of an object under investi- 
gation, and 5 
- collecting light emitted from the object with 
analyzing means (150, 350, 450, 650) with a 
detection aperture, spectrally dispersive ele- 
ments and a detector camera, 
wherein the illumination aperture and the io 
detection aperture are placed in conjugate opti- 
cal planes, 

characterized in that 

the light modulator means (120, 320, 420, 620) 
consisting of a two-dimensional array of light is 
modulator elements are controlled in a time- 
dependent fashion such that modulator ele- 
ments shaped according to the illumination 
aperture form an illumination pattern 
sequence. 20 

12. Method according to claim 11, wherein the light 
modulator elements are controlled such that at 
least a part of the object is scanned according to 
the predetermined pattern sequence with illumina- 25 
tion patterns being selected to be a systematically 
shifting line or lines either spaced regularly accord- 
ing to a pseudo-random sequence of finite length, 
according to a S-matrix type Hadamard sequence 

or randomly; regular dot lattices as a g. triangles, 30 
squares, rectangles, hexagons; random patterns or 
pseudo-random patterns of finite length based on 
so-called Walsch, Sylvester, Hadamard or Golay 
sequences; square or rectangular grids formed 
from intersecting line patterns; plane filling patterns 35 
being adapted to turn on the SLM elements an inte- 
gral number of times when sequentially produced 
on the SLM; or a combination of the above pattern 
sequences. 

40 

13. Method according to one of the claims 11 to 12, 
wherein simultaneously spectrally resolved conju- 
gate (i c ) and/or non-conjugate (l nc ) images of the 
object are collected with full field detector means. 

45 

14. Method according to one of the claims 11 to 13, 
wherein the step of focusing light includes a wave- 
length selection. 

1 5. Method according to one of the claims 1 3 or 1 4, fur- so 
ther comprising a deconvolution step wherein the 
object is reconstructed with a deconvolution algo- 
rithm combining the spectrally resolved conjugate 

(l c ) and non-conjugate (l nc ) images. 

55 

16. Method according to one of the claims 1 1 to 15, fur- 
ther comprising steps of simultaneous position 
selective initiating and monitoring photochemical 



reactions on a substrate, wherein the initiating 
includes the irradiation of predetermined sub- 
stances with a suitable wavelength and the monitor- 
ing includes fluorescence measurements with the 
substances or the reaction products. 

1 7. Method according to one of the foregoing claims 1 1 
to 16 used within: 

- investigations in the cell and tissue biology 

- analytical/biotechnological procedures, 
in-situ hybridization in genetic analysis, 

- formation of an optical mask for position selec- 
tive photochemical reactions, 

position selective spectroscopic measure- 
ments including excitation and emission spec- 
tra of one or more intrinsic or external 
fluorphores, lifetime measurements polariza- 
tion measurements and/or absorptions meas- 
urements in a reflection mode, 

- readout of biochemical arrays on chips, 

- large scale surface inspections in the semicon- 
ductor industry, and/or 

- optical recording and read-out. 

18. Programmable light source comprising a white or 
broad band light source (510), dispersion means 
(515) for wavelength-selection of light from the light 
source (510), spatial light modulator means (520) 
with a two-dimensional array of individually time- 
dependent controllable light modulator elements 
being illuminated with the dispersed light and pro- 
viding a position selective transmittiv'rty or reflectiv- 
ity, so that a light output with a predetermined 
wavelength distribution passes the spatial light 
modulator means (520). 

19. Programmable light source according to claim 18, 
further comprising a device for recombining the dis- 
persed wavelengths (530) being an integrating 
sphere, a lens system adapted for illumination in 
parallel mode (Kohl er type illumination), a matching 
set of dispersion optics, fiber optics, liquid light con- 
duits, or scattering surfaces or materials. 

20. Programmable light source according to claim 1 8 or 
19, wherein the modulator elements are controlla- 
ble with a transmission or reflection time depend- 
end pattern corresponding to rows or columns of an 
S-type Hadamard coding matrix or with a predeter- 
mined time sweep. 

21 . Programmable light source according to daim 1 8 or 
20 being the light source means of a confocal spec- 
tral imaging system according to one of the claims 
1 to 10. 
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